Supplementary material - S3: Behaviour sensitivity test

Introduction

This document presents the results assessing the impact of input assumptions on model behaviour for the Dcision Support tool for Child and Adolescence Obesity, a system dynamic model developed to explore underlying relationships that contribute to youth obesity.

Outline of behaviour sensitivity analysis

System dynamics models are described as being “causally-descriptive”, meaning that unlike other “black-box”, it is not sufficient that system dynamics only produce accurate output. In addition to being numerically valid, system dynamics models’ behaviour needs to be validated.

The test presented in this document examines which input assumptions have the greatest impact on model behaviour, unlike previous sensitivity analysis, which focuses on numerical relationships between input assumptions and outputs (either model outcomes or policy outcomes).

Methods

A sensitivity analysis was conducted testing 649 input variables between \(\pm\) 20% of an input’s initial value. While each input value may have more extreme values, these input values are considered to be outside of probable values. For each sensitivity analysis, the model generates 78 BMI prevalence outcomes, 3 BMI categories, 13 age groups and two genders.

Model behaviour is the trend, phase, lag or amplitude of a system dynamic model result over modelled time. Inputs that impact these outcome characteristics are considered to impact the model behaviour.

Outcome Measure

The behaviour of each model test was evaluated using the cumulative difference of the first derivative. The following plots explain how this outcome measure is calculated.

Figure panel A shows a plot of three outcomes, the base model, with no changes to any initial assumptions. The result varies an input assumption that produces similar model behaviour and a second test, where varying the input assumption produces different model behaviour. The absolute cumulative difference between the model outcomes is plotted in panel C.

Panel B plots the first derivative of each model result. The model results with similar behaviour also have a similar first derivative. Conversely, the different behaviour result has differences in the derivative. The cumulative absolute difference of the first derivative shows a greater difference between the two model results highlighting the difference between behaviours.

While the cumulative absolute difference measure cannot give insights into the model structure, it is useful to screen input assumptions that impact model behaviour .

Unlike the extreme test, this analysis focuses on determining which input assumption impacts the model behaviour.

Example plot for model behaviour

Example plot for model behaviour

How to interpret the results

Each input assumption is sequentially sampled 20 times between +/- 20% of the initial assumed input value. The cumulative difference of the first derivative is used to assess how each variable sample impacts model behaviour and the average cumulative difference between the 20 samples are summarised in the result plots.

The figure below steps through how the summary plots are calculated. The summary plot (subplot A) shows the average cumulative difference between the 20 samples of the metabolic equivalence of task (MET) for vigorous physical activity. This summary plot is presented for all tested variables.

The outcome, prevalence of obesity (subplot B), is plotted for each tested assumption of vigorous MET value across each age group. The subsequent derivative (subplot C) and cumulative difference in the derivative are also plotted (subplot D), with the average incremental difference plotted in red. This is the summary value presented in the summary plots.

A high average cumulative difference in the first derivative suggests a variable with a larger impact on model behaviour.

Results

Overall Ranking of sensitivity

The 649 input assumptions tested can be summarised into 70 variables by taking the mean of cumulative difference in derivatives age-gender-BMI specific inputs, resulting in 70 overarching variables. These are ranked in the following table.

Below is the ranking of aggregate input assumptions, where input assumptions with multiple input subgroups have been averaged. For example, changes in body weight are tested by age-gender-BMI subgroups however, are ranked based on the mean cumulative difference in first derivatives.

The results show that variables that have a global impact, i.e. those that impact all of the modelled population, have the greatest impact on model behaviour.

Label Average cumulative difference in derivatives
SUGAR Kj per gram 0.8977340
FATS Kj per gram 0.8944223
“NON-SUGAR CARBOHYDRATES Kj per gram” 0.8195918
INACTIVE METs 0.7031066
SLEEP METs 0.6844801
PROTEIN Kj per gram 0.5855636
SCREEN TIME METs 0.4285511
VIGOROUS PA METs 0.4048512
LIGHT PA METs 0.3383088
MODERATE PA METs 0.2971283
FATS TEF 0.1917250
CARBOHYDRATE TEF 0.1718966
PROTEIN TEF 0.1655541
Grains Reported Intake INTERCEPT 0.1599997
Dairy Reported Intake INTERCEPT 0.1541384
Discretionary foods Reported Intake INTERCEPT 0.1417191
Daily SLEEP minutes Reported INTERCEPT 0.1117347
Proportion of nutrients within food group Inputs 0.1012992
SUGAR TEF 0.0908475
Daily VIGOROUS PA minutes Reported INTERCEPT 0.0903123
Meat and Protein Reported Intake INTERCEPT 0.0832654
Discretionary foods Reported Intake AGE SLOPE 0.0822170
Years to achieve change 0.0802966
Schofield Equation Coefficient 0.0675960
“Body Weight (kg)” 0.0618553
Fruit Reported Intake INTERCEPT 0.0535116
Schofield Equation intercept 0.0478507
Daily MODERATE PA minutes Reported INTERCEPT 0.0462249
Meat and Protein Reported Intake AGE SLOPE 0.0421011
Grains Reported Intake AGE SLOPE 0.0385217
Other Reported Intake INTERCEPT 0.0372783
Vegetables Reported Intake INTERCEPT 0.0309869
Daily LIGHT PA minutes Reported INTERCEPT 0.0272870
Other Beverages Reported Intake INTERCEPT 0.0272579
Sugar based beverage Reported Intake AGE SLOPE 0.0261384
Sugar based beverage Reported Intake INTERCEPT 0.0252786
Fats and Oils Reported Intake INTERCEPT 0.0249171
Other Beverages Reported Intake AGE SLOPE 0.0147834
Other Reported Intake AGE SLOPE 0.0132773
Vegetables Reported Intake AGE SLOPE 0.0123815
Daily VIGOROUS PA minutes Reported AGE SLOPE 0.0113788
Initial BMI Prevalence Inputs 0.0095932
Dairy Reported Intake AGE SLOPE 0.0072566
Fats and Oils Reported Intake AGE SLOPE 0.0063582
Daily MODERATE PA minutes Reported AGE SLOPE 0.0063546
“Reference height (m)” 0.0053184
Fruit Reported Intake AGE SLOPE 0.0036740
Daily LIGHT PA minutes Reported AGE SLOPE 0.0035969
Daily SLEEP minutes Reported AGE SLOPE 0.0032736
“Growth Function kJ/day” 0.0031015
Adult to Child Social Transmission PAL Behaviors 0.0022373
Child to Child Social Transmission of PAL Behaviors 0.0018585
Parents BMI 0.0017146
Initial FM % 0.0009904
“Percentage BF >6mths reported” 0.0009100
Intercept 0.0004478
BMI Hazards Ratios 0.0002142
Adult to Adult Social Transmission of PAL Behaviors 0.0001529
“Percentage non-core > 0 reported” 0.0000676
Non-core >0 0.0000650
“Percentage TV >1 per day reported” 0.0000486
TV >=1 0.0000464
Breastfeeding >=6mth 0.0000272
Adult to Child Social Transmission DIET Behaviors 0.0000003
Child to Child Social Transmission of DIET Behavior 0.0000002
Adult to Adult Social Transmission of DIET Behaviors 0.0000000
Daily SCREEN TIME minutes Reported INTERCEPT 0.0000000
Daily SCREEN TIME minutes Reported AGE SLOPE 0.0000000
Water Reported Intake AGE SLOPE 0.0000000
Water Reported Intake INTERCEPT 0.0000000
Note:
Ranked variables

Examining the most sensitive variable: energy density of sugar

Body Weight

Change in Male Body Weight input assumptions

Change in Female Body Weight input assumptions

Height

Change in Male height input assumptions

Change in Female height input assumptions

Growth Function kJ/day

Macronutrient energy density

Thermic effect of food (TEF)

Adult to Adult Social Transmission

Adult to Child Social Transmission

Child to Child Social Transmission

Infant Reported behaviours

Assumed intergenerational relationships

Mortality ratios

The Metabolic Equivalent of Task (MET)

Light physical activity

Moderate physical activity

Vigorous physical activity

Screen time

Sleep

Fruit Reported Intake

Vegetables Reported Intake

Grains Reported Intake

Dairy Reported Intake

Meat and Protein Reported

Discretionary foods Reported

Fats and Oils Reported

Sugar based beverage Reported

Water Reported

Initial fat-mass (%)

Change in Male fat-mass % assumptions

Change in Female fat-mass % assumptions

Proportion of nutrients within food group Inputs

Grains

Vegetables

Fruit

Dairy

Meat and Protein

Fats

Discretionary foods

Sugar-sweetened beverages

Miscellaneous (Other)

Non-sugar-sweetened beverages

Initial BMI Prevalence Inputs

Change in Male Initial BMI Prevalence

Change in Female Initial BMI Prevalence

Plots for the chapter